User controlled device for sending control signals to an electric appliance, in particular user controlled pointing device such as mouse of joystick, with 3D-motion detection

ABSTRACT

A user controlled device, movable into a plurality of positions of a three-dimensional space, includes a MEMS acceleration sensor to detect 3D movements of the user controlled device. The device, such as a mouse, sends control signals correlated to the detected positions to an electrical appliance, such as a computer system. A microcontroller processes the output signals of the MEMS acceleration sensor to generate the control signals, such as screen pointer position signals and “clicking” functions.

BACKGROUND OF THE INVENTION

[0001] 1. Field of the Invention

[0002] The present invention relates to a user-controlled device, inparticular a mouse or a joystick, with 3D motion detection. Theinvention is however also applicable to game pads, trackballs and otherscreen pointing devices for a computer system as well as to devices forpointing or selecting predetermined tasks or information according totheir position, which are connected to a computer or acomputer-controlled system. The invention is also applicable to thecontrol of an electrical appliance, e.g., for switching on an electricalappliance and activate particular tasks, on the base of a 3D movementsignal generated by the user-controlled device.

[0003] 2. Description of the Related Art

[0004] As is known, mice are now the most common interface between aperson and a computer or a computer controlled device and arehand-displaced on a plane or two-dimensional surface to control a cursoror pointer or activate particular tasks. To this end, typical micecomprise a plurality of sensors detecting a 2D movement of the mouse; aplurality of buttons for entering commands and a communication interfacefor communication with the computer system.

[0005] In view of the ease of operation and spread in use of mice as aconvenient interface with computer systems, a number of functionalitiesare being developed to make mice still easier to use, to reduceoperation stresses and damages to arms and shoulders, to increase thenumber of tasks that may be controlled or selected through a mouse, toadapt to various specific requirement and operation environment or todetect movements with more degrees of freedom.

[0006] For example, a mouse has been proposed, having improved movementdetection capabilities, including detection of tilting in four differentdirections, rotation about its axis and a little vertical movement. Thismouse, described, e.g., in “The VideoMouse: A Camera-BasedMulti-Degree-of-Freedom Input Device,” by K. Hinckley et al., ACMUIST'99 Symposium on User Interface Software & Technology, CHI Letters 1(1), pp. 103-112, uses a video camera for detecting the movement.However, although the image processing systems are becoming cheaper andsmaller, the costs and dimensions of these systems do not allow theiruse in all systems. Furthermore, this type of movement detection has afunctionality highly dependent from light conditions and/or opticalfeatures of the surface the mouse rests on.

[0007] Furthermore, the known solutions not always allow operation bydisabled persons, having limited or no hand control.

BRIEF SUMMARY OF THE INVENTION

[0008] An embodiment of the invention improves a user controlled deviceof the indicated type, so as to allow a wider applicability.

[0009] According to various embodiments of the present invention, thereis provided a user controlled device, and a method for generatingcontrol signals.

[0010] According to an aspect of the invention, the user controlleddevice accommodates an accelerometer or acceleration sensor made withthe MEMS (MicroElectroMechanicalSystem) technology and able to detect 3Dmovements, in particular movements around two perpendicular axes, so asto sense the movement of the user controlled device in the space andsend corresponding control signals to an electrical appliance, e.g., acomputer system.

[0011] According to a first embodiment, the user controlled device issimilar to a conventional mouse with buttons, wheels and clickpossibilities, but instead of being configured so as to be displaceablein a plane, has a support allowing tilting of the device around twoperpendicular axes. The support may allow a vertical displacement of thedevice body.

[0012] According to another embodiment, the user-controlled device is anaid device for disabled individuals, in particular for persons having areduced residual mobility and can, e.g., only move the head. The controldevice has a housing accommodating a dual-axis MEMS accelerometeroperating as an inclinometer; the housing is worn by the person andfixed to a mobile limb or head. For example, the housing may be similarto the housings used for hearing aids, and may be supported in theeyeglass arm, or be fixed to the head through a hairband.

ACCORDING TO ANOTHER ASPECT OF THE INVENTION, THE CONTROL DEVICE ISCONNECTED THROUGH A WIRE OR IN A WIRELESS WAY TO THE COMPUTER SYSTEMBRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS

[0013] For the understanding of the present invention, preferredembodiments thereof are now described, purely as a non-limitativeexamples, with reference to the enclosed drawings, wherein:

[0014]FIG. 1 shows a perspective view of a first embodiment of thepresent control device;

[0015]FIG. 2 is a lateral view of the pointing device of FIG. 1;

[0016]FIG. 3 shows a perspective view of a second embodiment of thepresent control device;

[0017]FIG. 4 shows a perspective view of a third embodiment of thecontrol device;

[0018]FIG. 5 shows a perspective view of a fourth embodiment of thecontrol device;

[0019]FIG. 6 shows a perspective view of a fifth embodiment of thecontrol device;

[0020]FIG. 7 shows a perspective view of a sixth embodiment of thecontrol device;

[0021]FIG. 8 is a block diagram of the pointing device of FIGS. 1-7;

[0022]FIG. 9 is a different block diagram of the control device of FIGS.1-7;

[0023]FIG. 10 is another block diagram of the control device of FIGS.1-7; and

[0024]FIG. 11 is flow-chart of the operation of the pointing device ofFIG. 6 or 7.

DETAILED DESCRIPTION OF THE INVENTION

[0025]FIG. 1 shows a first embodiment of the present control device, inthe shape of a mouse 1. The mouse 1 comprises a body 2 of rounded shapehaving two buttons 3 and a central wheel 4 operable by the user, in aper se known manner. In the alternative, a middle button may be providedinstead of the wheel.

[0026] The body 2 is supported by a curved base 5 preferably in theshape of a spherical cap arranged with the convexity looking downwards,so as to allow tilting of the body 2 around two axes A and B,perpendicular to each other and to a vertical axis C. The curved base 5may be of rigid or resilient material (e.g., rubber), to allow avertical displacement of the body 2.

[0027] The body 2 accommodates the usual control circuitry (not shown),for detecting actuation of the buttons 3 (“clicking”) and rotation ofthe wheel 4 in a per se known manner and sending suitable signals to acomputer system (see FIGS. 7-9). Furthermore, the body 2 accommodates a3D-motion detection device 7 (the block diagram whereof is shown inFIGS. 7-9) based on a MEMS accelerometer detecting the acceleration andmovement of the body, in particular the tilting of the body 2 aroundaxes A and B and in case vertical displacement along the axis C andsending corresponding information or control signals toward the computersystem. The transmission circuitry may be in common with the usualbutton and wheel control circuitry.

[0028] In the shown embodiment, an electrical wire 8 connects the mouse1 with the computer system; in the alternative and in a per se knownmanner, the mouse 1 may be connected wireless (e.g., by optical or radiotransmission) to the computer system.

[0029] The mouse 1 is designed to be balanced and to rest in anhorizontal position in the absence of external forces and to tilt aroundaxis B (up-down movement of the front portion bearing the buttons 3 andthe wheel 4) and/or around axis A (left-right movement) under thepressure of a user's hand. The 3D-motion detection device 7 (asdiscussed in detail with reference to FIGS. 7-9) detects the tilting andin case the vertical displacement and generates corresponding signals tocontrol an arrow on a screen of the computer system. In particular, theleft-right mouse movement (around axis A) may cause a correspondingleft-right movement of an arrow on the screen; the up-down mousemovement (around axis B) may cause a corresponding up and down movementon the screen; the vertical movement may control further functions, forexample for handling more folders on the screen or performing otherpre-programmed dedicated tasks.

[0030]FIG. 3 shows a different embodiment, wherein the body 2 issupported by a suction cap 40 and a spring 41 is interposed between thesuction cap 40 and the body 2. By virtue of the spring 41, the body 2may be tilted around axes A and B and be displaced along axis C, and a3D-motion detection device (not shown) arranged inside the body 2 sendscorresponding control signals to a computer system. In the alternative,more springs, e.g., four springs, may be provided for a better tiltingcontrol, as schematically represented by dashed lines.

[0031]FIG. 4 shows another embodiment, wherein the body 2 is supportedby four balls 45, e.g., of rubber. Also here, the body 2 may perform atilting movement around axes A, B and vertical displacement alongvertical axis C, and a 3D-motion detection device (not shown) arrangedinside the body 2 sends corresponding control signals to a computersystem.

[0032]FIG. 5 shows another embodiment, wherein the control device is ajoystick 47, having usual buttons for function control and accommodatinga 3D-motion detection device 7.

[0033]FIGS. 6 and 7 show different embodiments specifically intended fordisabled persons. In this case, a pointing device 10 includes a housing15 fixedly attached to an article worn by the user. In FIG. 6, thehousing 15 is attached to a hairband 16 and accommodates the 3D-motiondetection device 7, shown with dashed lines; the pointing device may beconnected to the computer system through a wire 8 or, preferably, in awireless way.

[0034] In FIG. 7, the housing 15 is attached to an arm 18 of a pair ofglasses 19.

[0035] In both cases, tilting of the head of a user up-down and/orleft-right causes tilting of the pointing device around axes A, B, asvisible in the enlarged detail of FIG. 6, and sending of correspondingsignals to the computer system, analogously to the embodiments of FIGS.1-5. In addition, the pointing device 10 may be designed to associatespecific movements of the housing 15 to “click” functions, as belowdescribed in greater detail with reference to the flow-chart of FIG. 11.

[0036] A first embodiment of the 3D-motion detection device 7 is shownin FIG. 8. Here, the 3D-motion detection device 7 includes a 2-axisaccelerometer 20 manufactured in the MEMS technology, for example asdescribed in European patent application N. 02425320.5 filed on 21 May2002. Accelerometer 20 generates two output signals X, Y proportional tothe tilting angle of the mouse 1 or of the pointing device 10 aroundaxes A, B. Output signals X, Y are digitized in an analog-to-digitalconverter 21 and fed to a microcontroller 22. Microcontroller 22 mayalso receive further control signals, such as “clicking” signals fromthe buttons 3 and position signals from the wheel 4 in the embodimentsof FIGS. 1-5, as indicated with dashed line.

[0037] The 3D-motion detection device 7 also includes a communicationport 23, for example, an RS 232 or USB port for signal exchange betweenthe microcontroller 22 and a computer 25 including, i.e., a central unit26, a screen 27 and a keyboard 28.

[0038] In case of the mouse 1 or the pointing device 10 allow verticaldisplacement, a further accelerometer 29 may be provided to send acorresponding signal to AD converter 21 and microcontroller 22, as shownin dashed line.

[0039]FIG. 9 shows a second embodiment of the 3D-motion detection device7 comprising, instead of a 2-axis accelerometer, two 1-axisaccelerometers 30 and, in case, a vertical axis accelerometer 29, allmanufactured in the MEMS technology. For example, the accelerometerdescribed in U.S. application Ser. No. 10/128,133 filed on Apr. 23, 2002or the accelerometer described in U.S. Pat. No. 5,955,668 may be used.In this case, the accelerometers 30 are arranged so that oneaccelerometer 30 detects tilting around axis A and the otheraccelerometer 30 detects tilting around axis B, while accelerometer 29detect displacement along axis C, and generate respective output signalsX, Y and Z.

[0040]FIG. 10 shows a third embodiment including a 2-axis accelerometer20 as in FIG. 8, but communication with the computer 25 is accomplishedthrough a radio-frequency link (e.g., using the Bluetooth technology).Here, the microcontroller 22 is connected with an RFtransmitter/receiver 31 which, through antennas not shown, communicateswith an interface 32 including an analogous RF transmitter/receiver 33and a digital port 34, e.g., an RS 232 or USB port, in turn connectedwith the computer 25.

[0041] In the case of the mouse 1, the 3D-motion detection device 7detects the tilting movements of the body 2 and controls the position ofan arrow or other pointer on the screen 27 on the basis of the outputsignals X and Y fed by the 2-axis accelerometer 20 or the 1-axisaccelerometers 30 in a manner analogous to conventional mice.

[0042] In case of vertical displacement, signal Z may be used accordingto the pre-programmed task.

[0043] In case of the pointing device 10, as said, the microcontroller22 may control not only the shifting of an arrow on the screen 27, butalso “clicking” functions, based, e.g., on the variation speed of theoutput signals X, Y fed by the accelerometer(s) 20, 30. Conveniently,the microcontroller 22 is able to discriminate among unintentional smallmovements (e.g., tremors) of the user's head, intentional movements ofbigger entity for pointer position control and rapid head movements for“clicking.”

[0044] To this end, the microcontroller compares the entity of thedetected movements (difference between the current and previous outputsignals X and Y, fed by the accelerometer) with own movement thresholds,to detect an arrow control movement, and with a click threshold, greaterthan the movement thresholds, to detect a clicking movement. Inpractice, the derivative of the signals along directions X and Y iscalculated and compared with thresholds for controlling the position ofthe arrow on the screen or for controlling clicking functions.

[0045] In particular, when an arrow control movement is detected (slowmovement of the head), the signs of the derivatives indicate thedirection (up, down, left, right) of the arrow movement with a speedwhich is a function of the amplitude of the derivative. When instead aclicking function is detected, the sign of the derivative of one outputsignal indicates simple clicking of the left or of the right button, andthe sign of the derivative of the other output signal indicatescontinuous pressure of the left button or interruption of the continuouspressure, as below described in detail.

[0046] The thresholds may be programmed by the user in a setup phase ofthe pointing device 10, as well as the functions associated with slow orrapid movement.

[0047] A flow-chart of the control program of the microcontroller 22 forthe embodiments of FIG. 6, 7 is shown in FIG. 11.

[0048] Initially, step 50, thresholds THC, THX, THY, and constants Kx,Ky are initialized. Threshold THC represents the clicking threshold,that is the minimum derivative in absolute value for controlling aclicking function; THX represents the X-signal threshold, that is theminimum derivative in absolute value for recognizing a valid movementalong the X axis and THY represents the Y-signal threshold, that is theminimum derivative in absolute value for recognizing a valid movementalong the Y axis. Kx and Ky represent the desired movement speed.

[0049] Then, the (digitized) output signals X and Y from theaccelerometer(s) 20, 30 are read, step 52; the entity of the movement inthe X direction is calculated as difference between the output signal Xand a previous value XOLD and compared with positive clicking thresholdTHC, step 54. If the difference X-XOLD is higher than the positiveclicking threshold THC, a right click (corresponding to clicking of theright button in a conventional mouse) is detected and a correspondingsignal is sent to the computer system, step 56; otherwise the differenceX-XOLD is compared with the negative clicking threshold −THC, step 58.If the difference X-XOLD is lower than the negative clicking threshold−THC, a left click (corresponding to clicking of the left button in aconventional mouse) is detected and a corresponding signal is sent tothe computer system, step 60.

[0050] If the difference X-XOLD is higher than negative clickingthreshold −THC but lower than positive clicking threshold THC, output NOfrom step 58, the absolute value of the difference X-XOLD is comparedwith X-signal threshold THX to discriminate between an intentional smallmovement and a control movement, step 62. If the absolute value of thedifference X-XOLD is higher than X-signal threshold THX, a new positionX_POS of the mouse on the screen is calculated by adding a quantityKx*X, proportional to the detected output signal X, to the previousposition OLDX_POS and a corresponding signal is sent to the computersystem, step 64.

[0051] If the X-direction movement is lower than the X-signal thresholdTHX (output NO from step 62), as well as after detecting a clickingfunction (after steps 56, 60) and after calculating the new positionX_POS (after step 64), the variation of the output signal Y is checked,analogously to what described for the X signal. Thus, the entity of themovement in the Y direction is calculated as difference between theoutput signal Y and a previous value YOLD and compared with positiveclicking threshold THC, step 66. If the difference Y-YOLD is higher thanthe positive clicking threshold THC, a command analogous to thecontinuous pressure of the left button in a conventional mouse isdetected and a corresponding signal is sent to the computer system, step68; otherwise the difference Y-YOLD is compared with the negativeclicking threshold −THC, step 70. If the difference Y-YOLD is lower thanthe negative clicking threshold −THC, a release command of the leftbutton is detected and a corresponding signal is sent to the computersystem, step 72.

[0052] If the difference Y-YOLD is higher than negative clickingthreshold −THC but lower than positive clicking threshold THC, output NOfrom step 70, the absolute value of the difference Y-YOLD is comparedwith Y-signal threshold THY, step 74. If the absolute value of thedifference Y-YOLD is higher than Y-signal threshold THY, a new positionY_POS of the mouse on the screen is calculated by adding a quantityKy*Y, proportional to the detected output signal Y, to a previousposition value OLDY_POS and a corresponding signal is sent to thecomputer system, step 76.

[0053] If the Y-direction movement is lower than the Y-signal thresholdTHY (output NO from step 74), as well as after detecting a continuousclicking or clicking release function (after steps 68, 72) and aftercalculating the new position Y_POS (after step 76), the previous valuesXOLD, YOLD, OLDX_POS and OLDY_POS are updated with the current values X,Y, X_POS and Y_POS, step 78.

[0054] The cycle continues until the pointing device is switched off.

[0055] The advantages of the present invention are clear from the above.In particular, it is outlined that the detection of a 3D movement by wayof an MEMS accelerometer causes the control device to be very versatileas regards application, features and operativity. In particular, thecontrol device may be implemented as a mouse, joystick, trackball,control pad or other control device for a screen cursor or for selectionamong a number of alternatives presented on a screen or other display.The device may be implemented to allow a simple actuation, also bypersons having reduced movement capabilities; and additional control maybe implemented by a same control device.

[0056] Furthermore, the implementation as a mouse requires an actuationspace smaller than the actual mice, since no planar movement on aresting surface is required. Furthermore, no mouse pad is needed, andthe present pointing device may be actuated on top of any surface,independently from the texture or optical properties thereof.

[0057] The control device with 3D-movement detection by MEMSaccelerometers manufactured using semiconductor technologies is cheaperthan other prior solutions.

[0058] The device may be used to control actuation of differentoperations or tasks of an electrical appliance, which is veryadvantageous for disabled persons or in case that the user shouldrequire the hands free for other activities.

[0059] Finally, it is clear that numerous variations and modificationsmay be made to pointing, selection or, generally, control devicedescribed and illustrated herein, all falling within the scope of theinvention as defined in the attached claims.

[0060] In particular, the pointing, selection or control device may beimplemented in any support, such as any mouse, joystick, gamepad, PDA(personal digital assistant, allowing Web surfing, e-mail exchange andso on), mobile phone, that is 3D-movable or has a 3D-movable portion.

[0061] Furthermore, the shape of the support allowing tilting of thedevice body may vary; for example, in the embodiment of FIG. 4, theyieldable balls may be replaced by rigid balls connected to the body 2through elastic means.

[0062] All of the above U.S. patents, U.S. patent applicationpublications, U.S. patent applications, foreign patents, foreign patentapplications and non-patent publications referred to in thisspecification and/or listed in the Application Data Sheet, areincorporated herein by reference, in their entirety.

1. A user controlled device, comprising: a body, movable into aplurality of positions of a three-dimensional space; and a MEMSacceleration sensor coupled to the body and configured to detect 3Dmovements of said device and send control signals correlated to saidpositions to an electrical appliance.
 2. A device according to claim 1wherein said body is in a form selected between computer mouse,joystick, gamepad, and PDA.
 3. A device according to claim 1 whereinsaid body is supported so as to be tiltable around two axes.
 4. A deviceaccording to claim 3 wherein said body 2 is mouse-shaped and issupported by one of the following: rounded base, at least one spring,yieldable balls, or elastically supported balls.
 5. A device accordingto claim 1 wherein the body is a housing accommodating said MEMSacceleration sensor and fixed to an article to be worn by a disabledperson.
 6. A device according to claim 5 wherein said article isconfigured to be born by a user's head.
 7. A device according to claim 6wherein said article is selected between a hairband or eyeglasses.
 8. Adevice according to claim 1 wherein said MEMS acceleration sensorcomprises at least a two-axis accelerometer or two one-axisaccelerometers.
 9. A device according claim 1 wherein said MEMSacceleration sensor generates position signals, the device furthercomprising a microcontroller for generating said control signals fromsaid position signals.
 10. A device according to claim 9, furthercomprising transmission means for connecting said microcontroller tosaid electrical appliance, said transmission means including a wiretransmissive medium and a communication port.
 11. A device according toclaim 9, further comprising transmission means for connecting saidmicrocontroller to said electrical appliance, said transmission meansincluding an RF communication port for allowing wireless communicationwith said electrical appliance.
 12. A device according to claim 9wherein said microcontroller generates a first control signal fordisplacing a screen pointer along a first direction upon detection oftilting of said device around a first axis and a second control signalfor displacing said screen pointer along a second direction upondetection of tilting of said device around a second axis.
 13. A deviceaccording to claim 9 wherein said microcontroller 22 comprises means fordetecting activation of predetermined functions upon detectingvariations of said position signals greater than first presetthresholds.
 14. A device according to claim 13 wherein said functionsinclude clicking functions; continuous clicking; and continuous clickingrelease.
 15. A device according to claim 13 wherein said microcontrollergenerates said control signals only if said variations are higher thansecond preset thresholds, and are lower than said first presetthresholds.
 16. A method for generating control signals correlated topositions of a user controlled device, comprising: moving the usercontrolled device into one of a plurality of positions of athree-dimensional space; sending corresponding control signals to anelectrical appliance; and detecting 3D movements of said device througha MEMS acceleration sensor.
 17. A method according to claim 16, furthercomprising detecting tilting movements of a body accommodating said MEMSacceleration sensor around two axes.
 18. A method according to claim 16wherein said MEMS acceleration sensor generates position signalscorresponding to movements of said device along respective directions,and wherein said position signals are processed to generate said controlsignals.
 19. A method according to claim 18, further comprisingcomparing said position signals with respective first thresholds andgenerating said control signals upon detecting that said positionsignals exceed said respective first thresholds.
 20. A method accordingto claim 18 wherein said control signals comprise displacing signals fordisplacing a screen pointer along two directions.
 21. A method accordingto claim 18, further comprising comparing said position signals withrespective second thresholds, greater than said first thresholds, andgenerating function activation signals upon detecting that said positionsignals exceed said respective second thresholds.
 22. A method accordingclaim 21 wherein said function activation signals comprise clickingfunctions, continuous clicking, and continuous clicking release.
 23. Adevice, comprising: a body; a MEMS sensor coupled to the body, the MEMSsensor configured to produce position signals corresponding to movementof the body; and means for converting the position signals of the MEMSsensor to control signals suitable for controlling position of a pointeron a computer screen.
 24. The device of claim 23 wherein the MEMS sensoris configured to detect movements of the body along first and secondaxes.
 25. The device of claim 23 wherein the MEMS sensor is configuredto detect movements of the body around first and second axes.
 26. Thedevice of claim 23 wherein the means for converting includes means for:producing a first control signal suitable for moving the pointer in afirst direction in response to movement of the body in a correspondingfirst direction, and producing a second control signal suitable formoving the pointer in a second direction in response to movement of thebody in a corresponding second direction.
 27. The device of claim 26wherein the means for converting produces the first or second controlsignal only if the movement of the body in the respective correspondingdirection exceeds a first selected threshold.
 28. The device of claim 27wherein the means for converting includes means for producing functionsignals if the movement of the body in the respective correspondingdirection exceeds a second threshold.
 29. The device of claim 28 whereinthe function signals include a signal suitable for initiating a clickfunction, a continuous click function, and a continuous click releasefunction.
 30. A method, comprising: employing a MEMS sensor to detectmovement of a body in a three-dimensional space; producing movementsignals corresponding to the movement of the body; and producing controlsignals corresponding to the movement signals, suitable for moving apointer on a computer screen.
 31. The method of claim 30 wherein theproducing control signals step includes ignoring movement signals thatdo not exceed a first selected threshold.
 32. The method of claim 31,further comprising producing function signals corresponding to movementsignals that exceed a second threshold.
 33. The method of claim 30wherein the movement signals includes a first movement signalcorresponding to a movement of the body in a first direction and asecond movement signal corresponding to a movement of the body in asecond direction, and wherein the control signals includes a firstcontrol signal corresponding to the first movement signal and a secondcontrol signal corresponding to a second control signal.
 34. The methodof claim 33 wherein the first control signal is suitable for moving thepointer in a first pointer direction and the second control signal issuitable for moving the pointer in a second pointer direction.
 35. Themethod of claim 30, further comprising: coupling the body to a user'shead; and moving the body by moving the user's head.
 36. A system,comprising: a computer screen; a CPU coupled to the computer screen; apointing device coupled to the CPU; a MEMS sensor coupled to thepointing device, the MEMS sensor configured to produce position signalscorresponding to movement of the pointing device; and a microprocessorcoupled to the MEMS sensor configured to convert the position signals ofthe MEMS sensor to control signals suitable for controlling position ofa pointer on the computer screen.